fix: cache only texlive-fonts-extra installed files, install other LaTeX packages normally#14414
fix: cache only texlive-fonts-extra installed files, install other LaTeX packages normally#14414
Conversation
Signed-off-by: skjnldsv <skjnldsv@protonmail.com>
a410c48 to
63a1e50
Compare
|
@copilot, maybe we don't need to cache the other latex deps after all, since only the texlive-fonts-extra is the most problematic one. I think we can just install them normally on the build step. But we keep the manual install/restore of |
… keep only texlive-fonts-extra file cache Agent-Logs-Url: https://github.com/nextcloud/documentation/sessions/3cf79ce0-c254-4113-bab0-64e3ce8f361f Co-authored-by: skjnldsv <14975046+skjnldsv@users.noreply.github.com>
Done in 0d1af66. Dropped the entire |
The
texlive-fonts-extrapackage is the primary bottleneck when installing LaTeX dependencies in CI, due to its large size (~500 MB of font data). This PR handles it separately via a manual copy/restore cache, while all other LaTeX packages are installed normally viaapt-get install.Changes Made
setup-latex-cachejob — now exclusively handlestexlive-fonts-extra:~/.cache/texlive-fonts-extra)texlive-fonts-extravia apt, then copies all its files (enumerated viadpkg -L) into the cache directorybuildjob:texlive-fonts-extrainstalled-files cache and copies files back viasudo cp -r ~/.cache/texlive-fonts-extra/. /(with apt fallback if cache is missing)texlive-fonts-recommended,latexmk,texlive-latex-extra,texlive-latex-recommended,texlive-xetex,texlive-fonts-extra-links,xindy,tex-gyre) normally via a plainapt-get installwith retry logic.debapt-cache machinery (no moreConfigure apt cache,Download LaTeX packages,Restore LaTeX apt cache,Install LaTeX from cachesteps)☑️ Resolves
🖼️ Screenshots
N/A — CI workflow change only.
Original prompt
Problem
The
Install LaTeX from cachestep in.github/workflows/sphinxbuild.ymlis too slow — it hits a 5-minute timeout becausedpkg -iunpacking all texlive.debfiles from the apt cache takes too long. The current approach caches.debfiles but still re-runsdpkgon every job.Solution
Replace the current
.deb-based cache approach with a cached installed files approach:apt-get, copy the installed files to~/.cache/latex-installed/(a home-relative path safe foractions/cache)sudo cpactions/cacheaction work without permission issues (it only touches the home path), while a simplesudo cphandles the privileged restoreImplementation
Modify
.github/workflows/sphinxbuild.yml:In the
setup-latex-cachejobReplace the current apt download-only +
.debcache approach with:apt-get install(not--download-only)~/.cache/latex-installed/covering all relevant paths:/usr/share/texlive→~/.cache/latex-installed/texlive/usr/share/texmf→~/.cache/latex-installed/texmf/usr/bin/(latexmk, xelatex, pdflatex, lualatex, xindy, etc.) — usedpkg -Lon each package to find the exact binaries/etc/texmf/→~/.cache/latex-installed/etc-texmfdpkg -Loutput)chown -R $(id -u):$(id -g)the cache dir soactions/cachecan save it~/.cache/latex-installed/path withactions/cache/saveIn the
buildjobReplace the current "Restore LaTeX apt cache" + "Install LaTeX from cache" steps with:
~/.cache/latex-installed/viaactions/cache/restoresudo cp -rto put everything back in system pathslatexmk,xelatex,pdflatex, andxindyKey requirements
dpkg -Lfor all installed packages to get the complete, accurate file list — do not guess pathssetup-latex-cachejob should still uselookup-onlyon first check to skip re-running if cache already existsdpkgunpack on every buildtimeout-minutes: 5that was causing failures.debcache paths and related steps entirelyThe cache size may be large (~1-2GB) — that's acceptable given the speed improvement.
The following is the prior conversation context from the user's chat exploration (may be truncated):
User: Is it possible to have the workflow update the runner image by installing dependencies, and then use that image in the following jobs ? Because it seems even installing the packages takes too much time/cpu
Assistant: ## Short answer: Not natively — but there's a better approach
GitHub Actions jobs always start from a fresh runner image — you can't mutate a runner in one job and pass that modified state to the next. Each job is an isolated VM.
The right solution for your use case is a custom Docker image stored in GHCR with LaTeX pre-installed. This avoids the install overhead entirely since the packages are baked into the image layer.
The real problem
Looking at the failing job, the
Install LaTeX from cachestep is hitting a 5-minute timeout while runningsudo dpkg -i --force-dependson all the texlive.debfiles. That's a lot of large packages to unpack sequentially.Recommended fix: Pre-built Docker image on GHCR
Create a
Dockerfileand a separate workflow that builds + publishes it to GHCR. Then use it as the container in yourbuildjobs.1. Add a
Dockerfile:2. Add a workflow to build & publish the image: